In this notebook, we will apply the linear model approach presented in (1_simple_example) to a real-life data set.

# load packages
library(tidyverse)
library(biobroom)
library(MSnbase)
# set up standardised plotting scheme
theme_set(theme_bw(base_size = 20) +
            theme(panel.grid.major=element_blank(),
                  panel.grid.minor=element_blank(),
                  aspect.ratio=1))
cbPalette <- c("#E69F00", "#56B4E9", "#009E73", "#F0E442", "#0072B2", "#D55E00", "#CC79A7", "#999999")

We start by reading in the data. Our input here is the protein-level quantification for the Nocodazole arrest/release experiment conducted for the OOPS NBT paper. In this experiment, we wanted to assess changes in RNA binding in arrested/released cells. To do this, we quantified “total” protein abundance and RNA-bound (extracted by OOPS) protein abundance. The peptide-level abundances have been aggregated to protein level abundance and center-median normalised. Proteins with missing values have been removed.

total_protein_quant <- readRDS("../raw/total_res_pro_agg_norm.rds")
rbp_protein_quant <- readRDS("../raw/rbp_res_pro_agg_norm.rds")

The input data are in MSnSets. As a reminder, the MSnSet class mimics the ExpressionSet class and contains 3 matrices: 1. assay data (obtained via: exprs) 2. feature data (fData) 3. phenotype data (pData)

The assay data is the quantification of the features (PSMs/peptides/proteins) and contains one column per sample

The feature data describes each feature, e.g peptide sequence, master protein accession, retention time etc etc

The phenotype data describes the samples

Let’s take a look at our total protein quantification data. If we “print” the object, we get a summary including the processing steps performed.

Here we have 2761 features (proteins) quantified across 6 samples.

The varLabels describe the “Condition”, “Replicate” and “Type” for these samples. We’ll take a look at these in more detail shortly.

We can see that there were originally 20171 features which were combined into 18111 features using a user-defined function. This was the step at which peptides with the same sequence but different variable modifications were aggregated. Then, these 18111 features were combined into the 2761 features (peptide->protein aggregation). Finally, the data was center-median normalised and missing values with imputed with knn.

print(total_protein_quant)
MSnSet (storageMode: lockedEnvironment)
assayData: 2761 features, 6 samples 
  element names: exprs 
protocolData: none
phenoData
  sampleNames: Abundance.F1.127N.Sample Abundance.F1.127C.Sample ... Abundance.F1.129C.Sample (6 total)
  varLabels: Sample_name Condition Replicate Type
  varMetadata: labelDescription
featureData
  featureNames: A0AVT1 A0MZ66 ... Q9Y6Y8 (2761 total)
  fvarLabels: Checked Confidence ... CV.Abundance.F1.131.Sample (47 total)
  fvarMetadata: labelDescription
experimentData: use 'experimentData(object)'
Annotation:  
- - - Processing information - - -
Subset [20171,10][20171,10] Thu Aug  9 16:17:13 2018 
Combined 20171 features into 18111 using user-defined function: Thu Aug  9 16:17:32 2018 
Combined 18111 features into 2761 using user-defined function: Thu Aug  9 16:17:46 2018 
Normalised (center.median): Thu Aug  9 16:17:51 2018 
Data imputation using knn Thu Aug  9 16:17:58 2018 
  Using default parameters 
Subset [2761,10][2761,6] Wed Jun 12 15:16:25 2019 
 MSnbase version: 2.4.2 

Here’s the top of the assay data

print(dim(exprs(total_protein_quant)))
[1] 2761    6
print(head(exprs(total_protein_quant), 2))
       Abundance.F1.127N.Sample Abundance.F1.127C.Sample Abundance.F1.128N.Sample Abundance.F1.128C.Sample
A0AVT1                 5.379489                 5.296457                 5.356980                 5.444870
A0MZ66                 4.926415                 4.876517                 4.955341                 4.735426
       Abundance.F1.129N.Sample Abundance.F1.129C.Sample
A0AVT1                 5.532511                 5.411776
A0MZ66                 4.886217                 4.834322

… and the associated feature data. Notice that there are many columns in the feature data. These are all the additional columns output from PD in addition to the quantification. They are all stored here in case they are required.

print(head(fData(total_protein_quant), 2))

… and here is the phenotype data. As we can see, we have 3 replicates each of “M”, “G1” and “S” phase, plus an additional Control sample. For our purposes, we’re only going to be interested in the M and G1 phases so we can remove the other data. Both the total and RBP quantification objects have the exact same order

print(pData(total_protein_quant))

To detect changes in RNA binding, we can only consider RBPs where we have also quantified the total protein. Below, we identify these cases by intersecting the rownames of each MSnSet (the protein names)

intersecting_proteins <- intersect(rownames(total_protein_quant), rownames(rbp_protein_quant))
cat(sprintf("Out of a total of %s RBPs quantified,\nwe have total protein quantification for %s proteins = %s %%",
            length(rownames(rbp_protein_quant)),
            length(intersecting_proteins),
            round(100*length(intersecting_proteins)/length(rownames(rbp_protein_quant)), 2)))
Out of a total of 2149 RBPs quantified,
we have total protein quantification for 1916 proteins = 89.16 %

Below, we convert the MSnSet into a “tidy” format data.frame using biobroom::tidy()

total_exprs <- total_protein_quant[intersecting_proteins,] %>% # subset to intersecting proteins
  tidy(addPheno=TRUE)  %>% # "tidy" the object, e.g make it into a tidy data format --> long
  mutate(intensity=value) %>% dplyr::select(-value) # rename the "value" column -> "intensity"

Top of the total protein expression data.frame. See how each intensity value now has it’s own row with the other columns describing the associated aspects of the intensity value, e.g the protein and experimental condition

print(head(total_exprs))

Now we do the same for the RBP quantification and then concatenate the two data frames together.

oops_exprs <- rbp_protein_quant[intersecting_proteins,] %>%
  tidy(addPheno=TRUE)  %>%
  mutate(intensity=value) %>% dplyr::select(-value)
combined_exprs <- rbind(total_exprs, oops_exprs)

We want to tell R which is the order of the values in the condition and type columns so that the fold changes are in the expected direction, e.g positive = higher in G1 vs M.

combined_exprs$condition <- factor(combined_exprs$Condition, levels=c("M", "G1"))
combined_exprs$type <- factor(combined_exprs$Type, levels=c("Total", "OOPS"))

Now we model the protein intensity according to the models described in 1_simple_example_vd.Rmd. As an example, let’s see the results from just applying the models to a single UniprotID.

combined_exprs %>% filter(protein == 'A0AVT1') %>%
  ggplot(aes(Condition, intensity, group=1)) +
  geom_point(size=2) +
  stat_summary(geom="line", fun.y=mean) +
  xlab("") +
  facet_wrap(~type)

fit <- combined_exprs %>% filter(protein == 'A0AVT1') %>%
  lm(formula=intensity~Condition*Type)
print(summary(fit))

Call:
lm(formula = intensity ~ Condition * Type, data = .)

Residuals:
      Min        1Q    Median        3Q       Max 
-0.087537 -0.048708  0.007263  0.041451  0.069459 

Coefficients:
                     Estimate Std. Error t value Pr(>|t|)    
(Intercept)           6.53699    0.03620 180.590 9.89e-16 ***
ConditionM           -0.76710    0.05119 -14.985 3.88e-07 ***
TypeTotal            -1.07394    0.05119 -20.979 2.80e-08 ***
ConditionM:TypeTotal  0.64836    0.07240   8.956 1.92e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.0627 on 8 degrees of freedom
Multiple R-squared:  0.988, Adjusted R-squared:  0.9835 
F-statistic: 219.6 on 3 and 8 DF,  p-value: 5.07e-08

We can see that the model fits the data well (“Multiple R-squared: 0.9673, Adjusted R-squared: 0.955”). We can see that the interaction term that we’re interested in (for changes in RNA binding) significantly deviates from zero in both models.

Below, we make a function to run the linear models on a protein, select the best model and then return the required values from the model. When we run on the same protein as above, we can see that the best model is the one including the TMT tag as a co-variate.

Change_in_RNA_binding_LM <- function(obj, coeff_of_interest="conditionG1:typeOOPS"){
  
  fit <- obj %>% lm(formula=intensity ~ condition + type + condition*type)
  
  fit_values <- c(coef(summary(fit))[coeff_of_interest,],
                  summary(fit)$adj.r.squared)
  
  names(fit_values) <- c("lm_fold_change", "lm_std_error", "lm_t_value", "lm_p_value", "lm_adj_R_squared")
  fit_values <- as.data.frame(t(fit_values), stringsAsFactors=FALSE)
  
  return(fit_values)
}
combined_exprs %>% filter(protein == 'A0AVT1') %>% Change_in_RNA_binding_LM()

Below, we make a function to run the testModels() function on all proteins in turn using dplyr. We will use the standard Benjamini-Hochberg method to adjust p-values for the multiple tests we have conducted.


|===================================                                                          | 39% ~3 s remaining     
|====================================                                                         | 39% ~3 s remaining     
|====================================                                                         | 40% ~3 s remaining     
|=====================================                                                        | 41% ~3 s remaining     
|=====================================                                                        | 41% ~3 s remaining     
|=====================================                                                        | 41% ~3 s remaining     
|======================================                                                       | 41% ~3 s remaining     
|======================================                                                       | 42% ~3 s remaining     
|======================================                                                       | 42% ~4 s remaining     
|======================================                                                       | 42% ~4 s remaining     
|======================================                                                       | 42% ~4 s remaining     
|=======================================                                                      | 43% ~4 s remaining     
|========================================                                                     | 43% ~4 s remaining     
|========================================                                                     | 44% ~4 s remaining     
|=========================================                                                    | 44% ~4 s remaining     
|=========================================                                                    | 45% ~4 s remaining     
|==========================================                                                   | 46% ~4 s remaining     
|==========================================                                                   | 46% ~4 s remaining     
|===========================================                                                  | 46% ~4 s remaining     
|===========================================                                                  | 47% ~4 s remaining     
|============================================                                                 | 47% ~4 s remaining     
|============================================                                                 | 48% ~4 s remaining     
|=============================================                                                | 49% ~3 s remaining     
|=============================================                                                | 49% ~3 s remaining     
|==============================================                                               | 50% ~3 s remaining     
|===============================================                                              | 51% ~3 s remaining     
|===============================================                                              | 52% ~3 s remaining     
|================================================                                             | 52% ~3 s remaining     
|=================================================                                            | 53% ~3 s remaining     
|==================================================                                           | 54% ~3 s remaining     
|==================================================                                           | 55% ~3 s remaining     
|===================================================                                          | 56% ~3 s remaining     
|====================================================                                         | 56% ~3 s remaining     
|=====================================================                                        | 57% ~3 s remaining     
|======================================================                                       | 58% ~3 s remaining     
|======================================================                                       | 59% ~3 s remaining     
|=======================================================                                      | 60% ~3 s remaining     
|========================================================                                     | 61% ~3 s remaining     
|========================================================                                     | 61% ~3 s remaining     
|=========================================================                                    | 62% ~3 s remaining     
|==========================================================                                   | 63% ~3 s remaining     
|==========================================================                                   | 63% ~2 s remaining     
|===========================================================                                  | 64% ~2 s remaining     
|============================================================                                 | 65% ~2 s remaining     
|=============================================================                                | 66% ~2 s remaining     
|=============================================================                                | 66% ~2 s remaining     
|==============================================================                               | 67% ~2 s remaining     
|===============================================================                              | 68% ~2 s remaining     
|===============================================================                              | 68% ~2 s remaining     
|===============================================================                              | 68% ~2 s remaining     
|===============================================================                              | 69% ~2 s remaining     
|================================================================                             | 69% ~2 s remaining     
|=================================================================                            | 70% ~2 s remaining     
|==================================================================                           | 71% ~2 s remaining     
|==================================================================                           | 72% ~2 s remaining     
|===================================================================                          | 73% ~2 s remaining     
|===================================================================                          | 73% ~2 s remaining     
|====================================================================                         | 73% ~2 s remaining     
|====================================================================                         | 74% ~2 s remaining     
|=====================================================================                        | 74% ~2 s remaining     
|=====================================================================                        | 75% ~2 s remaining     
|=====================================================================                        | 75% ~2 s remaining     
|=====================================================================                        | 75% ~2 s remaining     
|=====================================================================                        | 75% ~2 s remaining     
|=====================================================================                        | 75% ~2 s remaining     
|======================================================================                       | 75% ~2 s remaining     
|======================================================================                       | 75% ~2 s remaining     
|======================================================================                       | 76% ~2 s remaining     
|======================================================================                       | 76% ~2 s remaining     
|=======================================================================                      | 77% ~2 s remaining     
|=======================================================================                      | 77% ~2 s remaining     
|=======================================================================                      | 77% ~2 s remaining     
|=======================================================================                      | 77% ~2 s remaining     
|=======================================================================                      | 77% ~2 s remaining     
|=======================================================================                      | 77% ~2 s remaining     
|========================================================================                     | 77% ~2 s remaining     
|========================================================================                     | 78% ~2 s remaining     
|=========================================================================                    | 78% ~2 s remaining     
|=========================================================================                    | 79% ~2 s remaining     
|=========================================================================                    | 79% ~2 s remaining     
|==========================================================================                   | 80% ~2 s remaining     
|==========================================================================                   | 80% ~2 s remaining     
|===========================================================================                  | 81% ~2 s remaining     
|============================================================================                 | 82% ~1 s remaining     
|=============================================================================                | 83% ~1 s remaining     
|=============================================================================                | 84% ~1 s remaining     
|==============================================================================               | 84% ~1 s remaining     
|===============================================================================              | 85% ~1 s remaining     
|===============================================================================              | 85% ~1 s remaining     
|===============================================================================              | 85% ~1 s remaining     
|===============================================================================              | 86% ~1 s remaining     
|================================================================================             | 87% ~1 s remaining     
|=================================================================================            | 88% ~1 s remaining     
|=================================================================================            | 88% ~1 s remaining     
|==================================================================================           | 89% ~1 s remaining     
|===================================================================================          | 90% ~1 s remaining     
|====================================================================================         | 90% ~1 s remaining     
|====================================================================================         | 91% ~1 s remaining     
|====================================================================================         | 91% ~1 s remaining     
|=====================================================================================        | 91% ~1 s remaining     
|=====================================================================================        | 92% ~1 s remaining     
|=====================================================================================        | 92% ~1 s remaining     
|======================================================================================       | 93% ~1 s remaining     
|=======================================================================================      | 94% ~1 s remaining     
|========================================================================================     | 95% ~0 s remaining     
|========================================================================================     | 95% ~0 s remaining     
|=========================================================================================    | 96% ~0 s remaining     
|==========================================================================================   | 97% ~0 s remaining     
|===========================================================================================  | 98% ~0 s remaining     
|===========================================================================================  | 99% ~0 s remaining     
|============================================================================================ |100% ~0 s remaining     

Below, we plot the p-values. Under the null hypothesis they should show an approximately uniform distribution. If there were a large number of proteins with a significant change in RNA binding, we would expect an additional “spike” with low p-values (<0.05). We see an approximately uniform distribution but with a slight skew towards low p-value. This may indicate the presence of changes in RNA binding but which we are insufficiently powered to detect, e.g low p-value but not significant low p-value.

M_G1 %>% ggplot(aes(lm_p_value)) + geom_histogram(bins=20)

So, we have detected a lot of proteins with a signficant change in RNA binding!!

We can use a volcano plot to take a look at the estimated fold changes and associated p-values

M_G1 %>%
  ggplot(aes(x=lm_fold_change, y=-log10(lm_p_value))) +
  geom_point(size=0.25)

We can make this volcano plot a bit more informative (and prettier) with a few extra lines:

M_G1 %>%
  mutate(sig=ifelse(lm_BH<0.01, "Sig.", "Not sig.")) %>% # add "sig" column
  ggplot(aes(x=lm_fold_change, y=-log10(lm_p_value), colour=sig)) +
  geom_point(size=0.25) + 
  scale_colour_manual(values=c("black", cbPalette[6]), name="") + # manually adjust colours
  xlab("Fold change (log2)") + ylab("-log10(p-value)") # manual axes labels

Finally, we save out the results for use in later notebooks

saveRDS(M_G1, "../results/M_G1_changes_in_RNA_binding_linear_model.rds")
LS0tCnRpdGxlOiAiSWRlbnRpZnlpbmcgY2hhbmdlcyBpbiBSTkEgYmluZGluZyB1c2luZyBhIGxpbmVhciBtb2RlbCIKb3V0cHV0OgogIHBkZl9kb2N1bWVudDogZGVmYXVsdAogIGh0bWxfbm90ZWJvb2s6IGRlZmF1bHQKaGVhZGVyLWluY2x1ZGVzOgotIFx1c2VwYWNrYWdle3hjb2xvcn0KLSBcdXNlcGFja2FnZXtmcmFtZWR9Ci0tLQoKClxjb2xvcmxldHtzaGFkZWNvbG9yfXtsaWdodGdyYXkhMTB9IAoKSW4gdGhpcyBub3RlYm9vaywgd2Ugd2lsbCBhcHBseSB0aGUgbGluZWFyIG1vZGVsIGFwcHJvYWNoIHByZXNlbnRlZCBpbiAoYDFfc2ltcGxlX2V4YW1wbGVgKSB0byBhIHJlYWwtbGlmZSBkYXRhIHNldC4KCmBgYHtyLCBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQojIGxvYWQgcGFja2FnZXMKbGlicmFyeSh0aWR5dmVyc2UpCmxpYnJhcnkoYmlvYnJvb20pCmxpYnJhcnkoTVNuYmFzZSkKCiMgc2V0IHVwIHN0YW5kYXJkaXNlZCBwbG90dGluZyBzY2hlbWUKdGhlbWVfc2V0KHRoZW1lX2J3KGJhc2Vfc2l6ZSA9IDIwKSArCiAgICAgICAgICAgIHRoZW1lKHBhbmVsLmdyaWQubWFqb3I9ZWxlbWVudF9ibGFuaygpLAogICAgICAgICAgICAgICAgICBwYW5lbC5ncmlkLm1pbm9yPWVsZW1lbnRfYmxhbmsoKSwKICAgICAgICAgICAgICAgICAgYXNwZWN0LnJhdGlvPTEpKQoKY2JQYWxldHRlIDwtIGMoIiNFNjlGMDAiLCAiIzU2QjRFOSIsICIjMDA5RTczIiwgIiNGMEU0NDIiLCAiIzAwNzJCMiIsICIjRDU1RTAwIiwgIiNDQzc5QTciLCAiIzk5OTk5OSIpCmBgYAoKV2Ugc3RhcnQgYnkgcmVhZGluZyBpbiB0aGUgZGF0YS4gT3VyIGlucHV0IGhlcmUgaXMgdGhlIHByb3RlaW4tbGV2ZWwgcXVhbnRpZmljYXRpb24gZm9yIHRoZSBOb2NvZGF6b2xlIGFycmVzdC9yZWxlYXNlIGV4cGVyaW1lbnQgY29uZHVjdGVkIGZvciB0aGUgT09QUyBOQlQgcGFwZXIuIEluIHRoaXMgZXhwZXJpbWVudCwgd2Ugd2FudGVkIHRvIGFzc2VzcyBjaGFuZ2VzIGluIFJOQSBiaW5kaW5nIGluIGFycmVzdGVkL3JlbGVhc2VkIGNlbGxzLiBUbyBkbyB0aGlzLCB3ZSBxdWFudGlmaWVkICJ0b3RhbCIgcHJvdGVpbiBhYnVuZGFuY2UgYW5kIFJOQS1ib3VuZCAoZXh0cmFjdGVkIGJ5IE9PUFMpIHByb3RlaW4gYWJ1bmRhbmNlLiBUaGUgcGVwdGlkZS1sZXZlbCBhYnVuZGFuY2VzIGhhdmUgYmVlbiBhZ2dyZWdhdGVkIHRvIHByb3RlaW4gbGV2ZWwgYWJ1bmRhbmNlIGFuZCBjZW50ZXItbWVkaWFuIG5vcm1hbGlzZWQuIFByb3RlaW5zIHdpdGggbWlzc2luZyB2YWx1ZXMgaGF2ZSBiZWVuIHJlbW92ZWQuCgpgYGB7cn0KdG90YWxfcHJvdGVpbl9xdWFudCA8LSByZWFkUkRTKCIuLi9yYXcvdG90YWxfcmVzX3Byb19hZ2dfbm9ybS5yZHMiKQpyYnBfcHJvdGVpbl9xdWFudCA8LSByZWFkUkRTKCIuLi9yYXcvcmJwX3Jlc19wcm9fYWdnX25vcm0ucmRzIikKYGBgCgpUaGUgaW5wdXQgZGF0YSBhcmUgaW4gTVNuU2V0cy4gQXMgYSByZW1pbmRlciwgdGhlIGBNU25TZXRgIGNsYXNzIG1pbWljcyB0aGUgYEV4cHJlc3Npb25TZXRgIGNsYXNzIGFuZCBjb250YWlucyAzIG1hdHJpY2VzOgoxLiBhc3NheSBkYXRhIChvYnRhaW5lZCB2aWE6IGBleHByc2ApCjIuIGZlYXR1cmUgZGF0YSAoYGZEYXRhYCkKMy4gcGhlbm90eXBlIGRhdGEgKGBwRGF0YWApCgpUaGUgYXNzYXkgZGF0YSBpcyB0aGUgcXVhbnRpZmljYXRpb24gb2YgdGhlIGZlYXR1cmVzIChQU01zL3BlcHRpZGVzL3Byb3RlaW5zKSBhbmQgY29udGFpbnMgb25lIGNvbHVtbiBwZXIgc2FtcGxlCgpUaGUgZmVhdHVyZSBkYXRhIGRlc2NyaWJlcyBlYWNoIGZlYXR1cmUsIGUuZyBwZXB0aWRlIHNlcXVlbmNlLCBtYXN0ZXIgcHJvdGVpbiBhY2Nlc3Npb24sIHJldGVudGlvbiB0aW1lIGV0YyBldGMKClRoZSBwaGVub3R5cGUgZGF0YSBkZXNjcmliZXMgdGhlIHNhbXBsZXMKCkxldCdzIHRha2UgYSBsb29rIGF0IG91ciB0b3RhbCBwcm90ZWluIHF1YW50aWZpY2F0aW9uIGRhdGEuIElmIHdlICJwcmludCIgdGhlIG9iamVjdCwgd2UgZ2V0IGEgc3VtbWFyeSBpbmNsdWRpbmcgdGhlIHByb2Nlc3Npbmcgc3RlcHMgcGVyZm9ybWVkLiAKCkhlcmUgd2UgaGF2ZSAyNzYxIGZlYXR1cmVzIChwcm90ZWlucykgcXVhbnRpZmllZCBhY3Jvc3MgNiBzYW1wbGVzLiAKClRoZSBgdmFyTGFiZWxzYCBkZXNjcmliZSB0aGUgIkNvbmRpdGlvbiIsICJSZXBsaWNhdGUiIGFuZCAiVHlwZSIgZm9yIHRoZXNlIHNhbXBsZXMuIFdlJ2xsIHRha2UgYSBsb29rIGF0IHRoZXNlIGluIG1vcmUgZGV0YWlsIHNob3J0bHkuIAoKV2UgY2FuIHNlZSB0aGF0IHRoZXJlIHdlcmUgb3JpZ2luYWxseSAyMDE3MSBmZWF0dXJlcyB3aGljaCB3ZXJlIGNvbWJpbmVkIGludG8gMTgxMTEgZmVhdHVyZXMgdXNpbmcgYSB1c2VyLWRlZmluZWQgZnVuY3Rpb24uIFRoaXMgd2FzIHRoZSBzdGVwIGF0IHdoaWNoIHBlcHRpZGVzIHdpdGggdGhlIHNhbWUgc2VxdWVuY2UgYnV0IGRpZmZlcmVudCB2YXJpYWJsZSBtb2RpZmljYXRpb25zIHdlcmUgYWdncmVnYXRlZC4gVGhlbiwgdGhlc2UgMTgxMTEgZmVhdHVyZXMgd2VyZSBjb21iaW5lZCBpbnRvIHRoZSAyNzYxIGZlYXR1cmVzIChwZXB0aWRlLT5wcm90ZWluIGFnZ3JlZ2F0aW9uKS4gRmluYWxseSwgdGhlIGRhdGEgd2FzIGNlbnRlci1tZWRpYW4gbm9ybWFsaXNlZCBhbmQgbWlzc2luZyB2YWx1ZXMgd2l0aCBpbXB1dGVkIHdpdGgga25uLiAKYGBge3J9CnByaW50KHRvdGFsX3Byb3RlaW5fcXVhbnQpCmBgYAoKCgpIZXJlJ3MgdGhlIHRvcCBvZiB0aGUgYXNzYXkgZGF0YQpgYGB7cn0KcHJpbnQoZGltKGV4cHJzKHRvdGFsX3Byb3RlaW5fcXVhbnQpKSkKcHJpbnQoaGVhZChleHBycyh0b3RhbF9wcm90ZWluX3F1YW50KSwgMikpCmBgYAoKLi4uIGFuZCB0aGUgYXNzb2NpYXRlZCBmZWF0dXJlIGRhdGEuIE5vdGljZSB0aGF0IHRoZXJlIGFyZSBtYW55IGNvbHVtbnMgaW4gdGhlIGZlYXR1cmUgZGF0YS4gVGhlc2UgYXJlIGFsbCB0aGUgYWRkaXRpb25hbCBjb2x1bW5zIG91dHB1dCBmcm9tIFBEIGluIGFkZGl0aW9uIHRvIHRoZSBxdWFudGlmaWNhdGlvbi4gVGhleSBhcmUgYWxsIHN0b3JlZCBoZXJlIGluIGNhc2UgdGhleSBhcmUgcmVxdWlyZWQuCmBgYHtyfQpwcmludChoZWFkKGZEYXRhKHRvdGFsX3Byb3RlaW5fcXVhbnQpLCAyKSkKYGBgCgouLi4gYW5kIGhlcmUgaXMgdGhlIHBoZW5vdHlwZSBkYXRhLiBBcyB3ZSBjYW4gc2VlLCB3ZSBoYXZlIDMgcmVwbGljYXRlcyBlYWNoIG9mICJNIiwgIkcxIiBhbmQgIlMiIHBoYXNlLCBwbHVzIGFuIGFkZGl0aW9uYWwgQ29udHJvbCBzYW1wbGUuIEZvciBvdXIgcHVycG9zZXMsIHdlJ3JlIG9ubHkgZ29pbmcgdG8gYmUgaW50ZXJlc3RlZCBpbiB0aGUgTSBhbmQgRzEgcGhhc2VzIHNvIHdlIGNhbiByZW1vdmUgdGhlIG90aGVyIGRhdGEuIEJvdGggdGhlIHRvdGFsIGFuZCBSQlAgcXVhbnRpZmljYXRpb24gb2JqZWN0cyBoYXZlIHRoZSBleGFjdCBzYW1lIG9yZGVyCmBgYHtyfQpwcmludChwRGF0YSh0b3RhbF9wcm90ZWluX3F1YW50KSkKYGBgCgpUbyBkZXRlY3QgY2hhbmdlcyBpbiBSTkEgYmluZGluZywgd2UgY2FuIG9ubHkgY29uc2lkZXIgUkJQcyB3aGVyZSB3ZSBoYXZlIGFsc28gcXVhbnRpZmllZCB0aGUgdG90YWwgcHJvdGVpbi4gQmVsb3csIHdlIGlkZW50aWZ5IHRoZXNlIGNhc2VzIGJ5IGludGVyc2VjdGluZyB0aGUgcm93bmFtZXMgb2YgZWFjaCBNU25TZXQgKHRoZSBwcm90ZWluIG5hbWVzKQpgYGB7cn0KaW50ZXJzZWN0aW5nX3Byb3RlaW5zIDwtIGludGVyc2VjdChyb3duYW1lcyh0b3RhbF9wcm90ZWluX3F1YW50KSwgcm93bmFtZXMocmJwX3Byb3RlaW5fcXVhbnQpKQoKY2F0KHNwcmludGYoIk91dCBvZiBhIHRvdGFsIG9mICVzIFJCUHMgcXVhbnRpZmllZCxcbndlIGhhdmUgdG90YWwgcHJvdGVpbiBxdWFudGlmaWNhdGlvbiBmb3IgJXMgcHJvdGVpbnMgPSAlcyAlJSIsCiAgICAgICAgICAgIGxlbmd0aChyb3duYW1lcyhyYnBfcHJvdGVpbl9xdWFudCkpLAogICAgICAgICAgICBsZW5ndGgoaW50ZXJzZWN0aW5nX3Byb3RlaW5zKSwKICAgICAgICAgICAgcm91bmQoMTAwKmxlbmd0aChpbnRlcnNlY3RpbmdfcHJvdGVpbnMpL2xlbmd0aChyb3duYW1lcyhyYnBfcHJvdGVpbl9xdWFudCkpLCAyKSkpCmBgYAoKCkJlbG93LCB3ZSBjb252ZXJ0IHRoZSBNU25TZXQgaW50byBhICJ0aWR5IiBmb3JtYXQgYGRhdGEuZnJhbWVgIHVzaW5nIGBiaW9icm9vbTo6dGlkeSgpYApgYGB7cn0KCnRvdGFsX2V4cHJzIDwtIHRvdGFsX3Byb3RlaW5fcXVhbnRbaW50ZXJzZWN0aW5nX3Byb3RlaW5zLF0gJT4lICMgc3Vic2V0IHRvIGludGVyc2VjdGluZyBwcm90ZWlucwogIHRpZHkoYWRkUGhlbm89VFJVRSkgICU+JSAjICJ0aWR5IiB0aGUgb2JqZWN0LCBlLmcgbWFrZSBpdCBpbnRvIGEgdGlkeSBkYXRhIGZvcm1hdCAtLT4gbG9uZwogIG11dGF0ZShpbnRlbnNpdHk9dmFsdWUpICU+JSBkcGx5cjo6c2VsZWN0KC12YWx1ZSkgIyByZW5hbWUgdGhlICJ2YWx1ZSIgY29sdW1uIC0+ICJpbnRlbnNpdHkiCgpgYGAKClRvcCBvZiB0aGUgdG90YWwgcHJvdGVpbiBleHByZXNzaW9uIGBkYXRhLmZyYW1lYC4gU2VlIGhvdyBlYWNoIGludGVuc2l0eSB2YWx1ZSBub3cgaGFzIGl0J3Mgb3duIHJvdyB3aXRoIHRoZSBvdGhlciBjb2x1bW5zIGRlc2NyaWJpbmcgdGhlIGFzc29jaWF0ZWQgYXNwZWN0cyBvZiB0aGUgaW50ZW5zaXR5IHZhbHVlLCBlLmcgdGhlIHByb3RlaW4gYW5kIGV4cGVyaW1lbnRhbCBjb25kaXRpb24KYGBge3J9CnByaW50KGhlYWQodG90YWxfZXhwcnMpKQpgYGAKXGNvbG9ybGV0e3NoYWRlY29sb3J9e3llbGxvdyExMH0gClxiZWdpbntzaGFkZWR9ClF1ZXN0aW9uOiBXaHkgZG9lc24ndCB0aGUgTVNuU2V0IG9iamVjdCBzdG9yZSBhbGwgdGhlIGRhdGEgaW4gdGhpcyBsb25nIGZvcm1hdD8KXGVuZHtzaGFkZWR9CgpcY29sb3JsZXR7c2hhZGVjb2xvcn17bGlnaHRncmF5ITEwfSAKCk5vdyB3ZSBkbyB0aGUgc2FtZSBmb3IgdGhlIFJCUCBxdWFudGlmaWNhdGlvbiBhbmQgdGhlbiBjb25jYXRlbmF0ZSB0aGUgdHdvIGRhdGEgZnJhbWVzIHRvZ2V0aGVyLgpgYGB7cn0Kb29wc19leHBycyA8LSByYnBfcHJvdGVpbl9xdWFudFtpbnRlcnNlY3RpbmdfcHJvdGVpbnMsXSAlPiUKICB0aWR5KGFkZFBoZW5vPVRSVUUpICAlPiUKICBtdXRhdGUoaW50ZW5zaXR5PXZhbHVlKSAlPiUgZHBseXI6OnNlbGVjdCgtdmFsdWUpCgpjb21iaW5lZF9leHBycyA8LSByYmluZCh0b3RhbF9leHBycywgb29wc19leHBycykKYGBgCgpXZSB3YW50IHRvIHRlbGwgUiB3aGljaCBpcyB0aGUgb3JkZXIgb2YgdGhlIHZhbHVlcyBpbiB0aGUgY29uZGl0aW9uIGFuZCB0eXBlIGNvbHVtbnMgc28gdGhhdCB0aGUgZm9sZCBjaGFuZ2VzIGFyZSBpbiB0aGUgZXhwZWN0ZWQgZGlyZWN0aW9uLCBlLmcgcG9zaXRpdmUgPSBoaWdoZXIgaW4gRzEgdnMgTS4KYGBge3J9CmNvbWJpbmVkX2V4cHJzJGNvbmRpdGlvbiA8LSBmYWN0b3IoY29tYmluZWRfZXhwcnMkQ29uZGl0aW9uLCBsZXZlbHM9YygiTSIsICJHMSIpKQpjb21iaW5lZF9leHBycyR0eXBlIDwtIGZhY3Rvcihjb21iaW5lZF9leHBycyRUeXBlLCBsZXZlbHM9YygiVG90YWwiLCAiT09QUyIpKQoKYGBgCgoKCk5vdyB3ZSBtb2RlbCB0aGUgcHJvdGVpbiBpbnRlbnNpdHkgYWNjb3JkaW5nIHRvIHRoZSBtb2RlbHMgZGVzY3JpYmVkIGluIGAxX3NpbXBsZV9leGFtcGxlX3ZkLlJtZGAuIEFzIGFuIGV4YW1wbGUsIGxldCdzIHNlZSB0aGUgcmVzdWx0cyBmcm9tIGp1c3QgYXBwbHlpbmcgdGhlIG1vZGVscyB0byBhIHNpbmdsZSBVbmlwcm90SUQuCmBgYHtyLCBvdXQud2lkdGggPSAnNzAlJ30KY29tYmluZWRfZXhwcnMgJT4lIGZpbHRlcihwcm90ZWluID09ICdBMEFWVDEnKSAlPiUKICBnZ3Bsb3QoYWVzKENvbmRpdGlvbiwgaW50ZW5zaXR5LCBncm91cD0xKSkgKwogIGdlb21fcG9pbnQoc2l6ZT0yKSArCiAgc3RhdF9zdW1tYXJ5KGdlb209ImxpbmUiLCBmdW4ueT1tZWFuKSArCiAgeGxhYigiIikgKwogIGZhY2V0X3dyYXAofnR5cGUpCgpmaXQgPC0gY29tYmluZWRfZXhwcnMgJT4lIGZpbHRlcihwcm90ZWluID09ICdBMEFWVDEnKSAlPiUKICBsbShmb3JtdWxhPWludGVuc2l0eX5Db25kaXRpb24qVHlwZSkKCnByaW50KHN1bW1hcnkoZml0KSkKCmBgYAoKV2UgY2FuIHNlZSB0aGF0IHRoZSBtb2RlbCBmaXRzIHRoZSBkYXRhIHdlbGwgKCJNdWx0aXBsZSBSLXNxdWFyZWQ6ICAwLjk2NzMsCUFkanVzdGVkIFItc3F1YXJlZDogIDAuOTU1ICIpLiBXZSBjYW4gc2VlIHRoYXQgdGhlIGludGVyYWN0aW9uIHRlcm0gdGhhdCB3ZSdyZSBpbnRlcmVzdGVkIGluIChmb3IgY2hhbmdlcyBpbiBSTkEgYmluZGluZykgc2lnbmlmaWNhbnRseSBkZXZpYXRlcyBmcm9tIHplcm8gaW4gYm90aCBtb2RlbHMuCgpCZWxvdywgd2UgbWFrZSBhIGZ1bmN0aW9uIHRvIHJ1biB0aGUgbGluZWFyIG1vZGVscyBvbiBhIHByb3RlaW4sIHNlbGVjdCB0aGUgYmVzdCBtb2RlbCBhbmQgdGhlbiByZXR1cm4gdGhlIHJlcXVpcmVkIHZhbHVlcyBmcm9tIHRoZSBtb2RlbC4gV2hlbiB3ZSBydW4gb24gdGhlIHNhbWUgcHJvdGVpbiBhcyBhYm92ZSwgd2UgY2FuIHNlZSB0aGF0IHRoZSBiZXN0IG1vZGVsIGlzIHRoZSBvbmUgaW5jbHVkaW5nIHRoZSBUTVQgdGFnIGFzIGEgY28tdmFyaWF0ZS4KYGBge3J9CkNoYW5nZV9pbl9STkFfYmluZGluZ19MTSA8LSBmdW5jdGlvbihvYmosIGNvZWZmX29mX2ludGVyZXN0PSJjb25kaXRpb25HMTp0eXBlT09QUyIpewogIAogIGZpdCA8LSBvYmogJT4lIGxtKGZvcm11bGE9aW50ZW5zaXR5IH4gY29uZGl0aW9uICsgdHlwZSArIGNvbmRpdGlvbip0eXBlKQogIAogIGZpdF92YWx1ZXMgPC0gYyhjb2VmKHN1bW1hcnkoZml0KSlbY29lZmZfb2ZfaW50ZXJlc3QsXSwKICAgICAgICAgICAgICAgICAgc3VtbWFyeShmaXQpJGFkai5yLnNxdWFyZWQpCiAgCiAgbmFtZXMoZml0X3ZhbHVlcykgPC0gYygibG1fZm9sZF9jaGFuZ2UiLCAibG1fc3RkX2Vycm9yIiwgImxtX3RfdmFsdWUiLCAibG1fcF92YWx1ZSIsICJsbV9hZGpfUl9zcXVhcmVkIikKICBmaXRfdmFsdWVzIDwtIGFzLmRhdGEuZnJhbWUodChmaXRfdmFsdWVzKSwgc3RyaW5nc0FzRmFjdG9ycz1GQUxTRSkKICAKICByZXR1cm4oZml0X3ZhbHVlcykKfQoKYGBgCgpgYGB7cn0KY29tYmluZWRfZXhwcnMgJT4lIGZpbHRlcihwcm90ZWluID09ICdBMEFWVDEnKSAlPiUgQ2hhbmdlX2luX1JOQV9iaW5kaW5nX0xNKCkKYGBgCgpCZWxvdywgd2UgbWFrZSBhIGZ1bmN0aW9uIHRvIHJ1biB0aGUgYHRlc3RNb2RlbHMoKWAgZnVuY3Rpb24gb24gYWxsIHByb3RlaW5zIGluIHR1cm4gdXNpbmcgZHBseXIuIFdlIHdpbGwgdXNlIHRoZSBzdGFuZGFyZCBCZW5qYW1pbmktSG9jaGJlcmcgbWV0aG9kIHRvIGFkanVzdCBwLXZhbHVlcyBmb3IgdGhlIG11bHRpcGxlIHRlc3RzIHdlIGhhdmUgY29uZHVjdGVkLgpgYGB7ciwgZWNobz1GQUxTRSwgd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRX0KcnVuTE0gPC0gZnVuY3Rpb24ob2JqLCBjb2VmZl9vZl9pbnRlcmVzdD0iY29uZGl0aW9uRzE6dHlwZU9PUFMiKXsKICByZXN1bHRzIDwtIG9iaiAlPiUKICAgIGdyb3VwX2J5KHByb3RlaW4pICU+JSAjIGdyb3VwIHRoZSBkYXRhIGZyYW1lIGJ5IHRoZSB1bmlxdWUgcHJvdGVpbiB2YWx1ZXMKICAgICMgQXBwbHkgdGhlIENoYW5nZV9pbl9STkFfYmluZGluZ19MTSBmdW5jdGlvbnMgdG8gZWFjaCBncm91cDsgIi4iIGhlcmUgaXMgdGhlIHdob2xlIGRhdGEgZnJhbWUgCiAgICBkbyhDaGFuZ2VfaW5fUk5BX2JpbmRpbmdfTE0oLiwgY29lZmZfb2ZfaW50ZXJlc3QpKSAKCiAgcmVzdWx0cyRsbV9CSCA8LSBwLmFkanVzdChyZXN1bHRzJGxtX3BfdmFsdWUsIG1ldGhvZD0iQkgiKSMgRkRSIAogIAogIHJldHVybihyZXN1bHRzKQogICAKfQoKTV9HMSA8LSBjb21iaW5lZF9leHBycyAlPiUgcnVuTE0oKQoKYGBgCgpCZWxvdywgd2UgcGxvdCB0aGUgcC12YWx1ZXMuIFVuZGVyIHRoZSBudWxsIGh5cG90aGVzaXMgdGhleSBzaG91bGQgc2hvdyBhbiBhcHByb3hpbWF0ZWx5IHVuaWZvcm0gZGlzdHJpYnV0aW9uLiBJZiB0aGVyZSB3ZXJlIGEgbGFyZ2UgbnVtYmVyIG9mIHByb3RlaW5zIHdpdGggYSBzaWduaWZpY2FudCBjaGFuZ2UgaW4gUk5BIGJpbmRpbmcsIHdlIHdvdWxkIGV4cGVjdCBhbiBhZGRpdGlvbmFsICJzcGlrZSIgd2l0aCBsb3cgcC12YWx1ZXMgKDwwLjA1KS4gV2Ugc2VlIGFuIGFwcHJveGltYXRlbHkgdW5pZm9ybSBkaXN0cmlidXRpb24gYnV0IHdpdGggYSBzbGlnaHQgc2tldyB0b3dhcmRzIGxvdyBwLXZhbHVlLiBUaGlzIG1heSBpbmRpY2F0ZSB0aGUgcHJlc2VuY2Ugb2YgY2hhbmdlcyBpbiBSTkEgYmluZGluZyBidXQgd2hpY2ggd2UgYXJlIGluc3VmZmljaWVudGx5IHBvd2VyZWQgdG8gZGV0ZWN0LCBlLmcgbG93IHAtdmFsdWUgYnV0IG5vdCBzaWduaWZpY2FudCBsb3cgcC12YWx1ZS4KYGBge3IsIG91dC53aWR0aCA9ICc1MCUnfQpNX0cxICU+JSBnZ3Bsb3QoYWVzKGxtX3BfdmFsdWUpKSArIGdlb21faGlzdG9ncmFtKGJpbnM9MjApCmBgYCAKClxjb2xvcmxldHtzaGFkZWNvbG9yfXtibHVlITEwfSAKXGJlZ2lue3NoYWRlZH0KVGFzazogSG93IG1hbnkgc2lnbmlmaWNhbnQgY2hhbmdlcyBpbiBSTkEgYmluZGluZyB3ZXJlIGRldGVjdGVkIChZb3UnbGwgbmVlZCB0byBzZXR0bGUgb24gYSBzdWl0YWJsZSBGRFIgdGhyZXNob2xkKSA/CgpcZW5ke3NoYWRlZH0KClxjb2xvcmxldHtzaGFkZWNvbG9yfXtsaWdodGdyYXkhMTB9IAoKClNvLCB3ZSBoYXZlIGRldGVjdGVkIGEgbG90IG9mIHByb3RlaW5zIHdpdGggYSBzaWduZmljYW50IGNoYW5nZSBpbiBSTkEgYmluZGluZyEhCgpXZSBjYW4gdXNlIGEgdm9sY2FubyBwbG90IHRvIHRha2UgYSBsb29rIGF0IHRoZSBlc3RpbWF0ZWQgZm9sZCBjaGFuZ2VzIGFuZCBhc3NvY2lhdGVkIHAtdmFsdWVzCmBgYHtyLCBvdXQud2lkdGggPSAnNTAlJ30KTV9HMSAlPiUKICBnZ3Bsb3QoYWVzKHg9bG1fZm9sZF9jaGFuZ2UsIHk9LWxvZzEwKGxtX3BfdmFsdWUpKSkgKwogIGdlb21fcG9pbnQoc2l6ZT0wLjI1KQoKYGBgCgpXZSBjYW4gbWFrZSB0aGlzIHZvbGNhbm8gcGxvdCBhIGJpdCBtb3JlIGluZm9ybWF0aXZlIChhbmQgcHJldHRpZXIpIHdpdGggYSBmZXcgZXh0cmEgbGluZXM6CmBgYHtyLCBvdXQud2lkdGggPSAnNTAlJ30KTV9HMSAlPiUKICBtdXRhdGUoc2lnPWlmZWxzZShsbV9CSDwwLjAxLCAiU2lnLiIsICJOb3Qgc2lnLiIpKSAlPiUgIyBhZGQgInNpZyIgY29sdW1uCiAgZ2dwbG90KGFlcyh4PWxtX2ZvbGRfY2hhbmdlLCB5PS1sb2cxMChsbV9wX3ZhbHVlKSwgY29sb3VyPXNpZykpICsKICBnZW9tX3BvaW50KHNpemU9MC4yNSkgKyAKICBzY2FsZV9jb2xvdXJfbWFudWFsKHZhbHVlcz1jKCJibGFjayIsIGNiUGFsZXR0ZVs2XSksIG5hbWU9IiIpICsgIyBtYW51YWxseSBhZGp1c3QgY29sb3VycwogIHhsYWIoIkZvbGQgY2hhbmdlIChsb2cyKSIpICsgeWxhYigiLWxvZzEwKHAtdmFsdWUpIikgIyBtYW51YWwgYXhlcyBsYWJlbHMKYGBgCgoKRmluYWxseSwgd2Ugc2F2ZSBvdXQgdGhlIHJlc3VsdHMgZm9yIHVzZSBpbiBsYXRlciBub3RlYm9va3MKYGBge3J9CnNhdmVSRFMoTV9HMSwgIi4uL3Jlc3VsdHMvTV9HMV9jaGFuZ2VzX2luX1JOQV9iaW5kaW5nX2xpbmVhcl9tb2RlbC5yZHMiKQpgYGAKCg==